A review of real-time segmentation of uncompressed video sequences for content-based search and retrieval

We present in this paper a review of methods for segmentation of uncompressed video sequences. Video segmentation is usually performed in the temporal domain by shot change detection. In case of real-time segmentation, computational complexity is one of the criteria which has to be taken into account when comparing different methods. When dealing with uncompressed video sequences, this criterion is even more significant. However previous published reviews did not involve complexity criterion when comparing shot change detection methods. Only recognition rate and ability to classify detected shot changes were considered. So contrary to previous reviews we give here complexity of most of the described methods. We review in this paper an extensive set of methods presented in the literature and classify them in several parts, depending on the information used to detect shot changes. The earliest methods were comparing successive frames by relying on most simple elements, that is to say pixels. Comparison could be performed on a global level, so methods based on histograms were also proposed. Block-based methods have been considered to process data at an intermediate level, between local (using pixels) and global (using histograms) levels. More complex features can be involved, resulting in feature-based methods. Alternatively some methods rely on motion as a criterion to detect shot changes. Finally different kinds of information could be combined together in order to increase the quality of shot change detection. So our review will detail segmentation methods ∗S. Lefèvre, J. Holler, N. Vincent, A Review of Real-Time Segmentation of Uncompressed Video Sequences for Content-Based Search and Retrieval, Real Time Imaging, Vol. 9, No. 1, 2003, pages 73-98, doi:10.1016/S1077-2014(02)00115-8